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Law of Large Numbers 


7.1.1 Law of Large Numbers 

The law of large numbers has a very central role in probability and statistics. It states that if you repeat an 
experiment independently a large number of times and average the result, what you obtain should be close to the 
expected value. There are two main versions of the law of large numbers. They are called the weak and strong 
laws of the large numbers. The difference between them is mostly theoretical. In this section, we state and prove 
the weak law of large numbers (WLLN). The strong law of large numbers is discussed in Section 7.2 . Before 
discussing the WLLN, let us define the sample mean. 


Definition 7.1. For i.i.d. random variables X\, X 2 , ■ ■ . , X n . the sample mean, denoted by X . is defined 
as 

_ X\ + X2+... -\~x n 

n 

Another common notation for the sample mean is M n . If the Xj's have CDF Fx (®), we might show the 
sample mean by M n (^C) to indicate the distribution of the Xj 's. 


Note that since the Xj 's are random variables, the sample mean, X = M n (X). is also a random variable. In 
particular, we have 


^ r —, EX 1 + EX 2 +...+EX n 

E[X] = -—-—-—-- 

n 

__ nEX 
n 

= EX. 


(by linearity of expectation) 
(since EXj = EX) 


Also, the variance of X is given by 

„ Var/X, + X 2 +... +X n ) 

Var(X) =-^- 

_ Var(Xi) + Var(X 2 )+... +Var(X n ) 
n 2 

nVar(X) 

n 2 

_ Var(X) 
n 


(since Var(aX) = a 2 Var(X)) 
(since the Xj s are independent) 
(since Var(X^) = Var(X)) 


Now let us state and prove the weak law of large numbers (WLLN). 


The weak law of large numbers (WLLN) 
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Let X\, X 2 ,..., X n be i.i.d. random variables with a finite expected value EX { = 
for any € > 0, 


lim P(\X — n\ > e) = 0. 

n—¥ 00 


• Proof 

o The proof of the weak law of large number is easier if we assume Var(^C) = 
case we can use Chebyshev's inequality to write 


P(\X - ij.\ > e) < 

__ Var(X) 
ne 2 ’ 


Var(X) 


which goes to zero as Tl —>■ 00. 
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< OO. Then, 


o 

(T is finite. In this 
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